Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning

The goal of few-shot learning is to learn a classifier that can recognize unseen classes from limited support data with labels. A common practice for this task train model on the base set first and then transfer novel through fine-tuning or meta-learning. However, as have no overlap set, simply transferring whole knowledge not an optimal solution since some in may be biased even harmful class. In paper, we propose partial by freezing particular layer(s) model. Specifically, layers will imposed different rates if they are chosen fine-tuned, control extent preserved transferability. To determine which recast what values them, introduce evolutionary search based method efficient simultaneously locate target their individual rates. We conduct extensive experiments CUB mini-ImageNet demonstrate effectiveness our proposed method. It achieves state-of-the-art performance both meta-learning non-meta frameworks. Furthermore, extend conventional pre-training + paradigm obtain consistent improvement.